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divided into three kinds of fixed, adaptive and fully adaptive ones. They have proved 
exponential time lower bounds of exact and approximation algorithms under adap- 
tive BT model for Knapsack problem. Their exact lower bound is il(2°' 5n /y / n), in 
■ this paper, we slightly improve the exact lower bound to about Q(2 omn /^/n), by 
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Abstract 

M.Aleknovich et al. have recently proposed a model of algorithms, called BT model, 
which generalizes both the priority model of Borodin, Nielson and Rackoff, as well 
as a simple dynamic programming model by Woeginger. BT model can be further 



the same technique, with related parameters optimized. 



1 Introduction to Backtracking model 



M.Aleknovich et. al have proposed a backtracking model in [I] which gener- 
alizes both the priority model of Borodin, Nielson and Rackoff [6 J , as well as 
a simple dynamic programming model due to Woeginger [TJ, and hence spans 
a wide spectrum of algorithms. Borodin then shared his insights into these 
models in [8]. The definition of backtracking model is as follows. 

Let T> be an arbitrary data domain that contains objects Di called data items. 
Let if be a set, representing the set of allowable decisions for a data item. For 
example, for the knapsack problem, a natural choice for T> would be the set of 
all pairs (x,p) where a; is a weight and p is a profit; the natural choice for H 
is 0, 1 where is the decision to reject an item and 1 is the decision to accept 
an item. 

A Backtracking search/optimization problem P is specified by a pair (T>p, fp) 
where T>p is the underlying domain, and fp is a family of objective functions, 
fp : (Di, D n , ax, a n ) h- > TZ, where ax,---,a n is a set of variables that 
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range over H, and D±, D n is a set of variables that range over D. On input 
I = Dx, ...D„ G T>, the goal is to assign each a, a value in H so as to maximize 
(or minimize) fp. A search problem is a special case where fp 1 outputs either 
or 1. 

For any domain S write O(S) for the set of all orderings of elements of S. 

Definition 1 A backtracking algorithm A for problem P = (T>, {/"}) consists 
of the ordering functions 

r\ : V k x H k ^ OiV) 

and the choice functions 

c\ : V k+1 x^h 0(HU {_L}, 

where k = 0, n — 1. 

There are three classes of BT algorithms. 

• Fixed algorithms: does not depend upon any of its arguments. 

• Adaptive algorithms: r\ depends on D\, but not on a%, a^. 

• adaptive algorithms: r\ depends on both Di, and ai, a^. 

The value of r k A specifies the order to consider the remaining items, given that 
the choices about the first k items have been made; the value of c k A specifies the 
order to make possible decisions about Z?fc+i- F° r m ore detailed explanation 
of BT model, refer to pp. But it seems reasonable to repeat the definition of 
computation tree here, which serves as a good interpretation of backtracking. 

Definition 2 Assume that P is a BT problem and A is a BT algorithm for 
P , For any instance I = (Di, D n ), Di G Vp we define the computation tree 
Ta{I) as an oriented rooted tree in the following recursive way. 

• Each node v of depth k in the tree is labelled by a tuple (D%, D%, a\, afc). 

• The root node has the empty label. 

• For every node v of depth k < n with a label (D v ,~a v ), let D1 +l be the 
data item in I\{D\, ...,D%} that goes first in the list r\(D v \~a v ). Assume 
that the output c A (D v , D% +1 , ~~a v ) has the form (ci, Cj, J_, Cd+i, ■■■), where 
c,i G H . If d = then v has no children. Otherwise it has d child nodes 
v i, Vd that go from left to right and have labels 
(Z^...,Z^<V--XVi) = Pr,...,^ ! +1 ,<%...,<% Q ) resp. 

Following is an example diagram of computation tree, nodes for data items, 
edges for choices. It's easy to see that it represents an adaptive BT algorithm, 
but not fully adaptive. 
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Fig. 1. A computation tree of some adaptive BT algorithm 

Clearly the width of a computation tree is a lower bound of the computa- 
tional complexity of the corresponding BT algorithm, so we will consider the 
asymptotic lower bounds of the computation tree width in the following text. 



2 General lower bound strategy 



To understand the proof shown later, we give the basic idea to prove such 
lower bounds first. It's a game between Solver and Adversary. Initially, the 
Adversary presents to the algorithm(Solver) some finite set of possible input 
items, Po, and the partial instance PIq is empty, T is the set consisting of 
the null partial solution. The game consists of a series of phases. At any phase 
i, there is a set of possible data items Pj, a partial instance Pli and a set T; 
of partial solutions for P/j. In phase %, % > 1, the Solver picks any data item 
a G Pi-i, adds a to obtain PI^ = Pli-\ U {a}, and chooses a set Tj of partial 
solutions, each of which must extend a solution in The Adversary then 
removes a and some further items to obtain Pj. 

The strategy of Adversary will usually have the following form. The number 
of rounds, n will be fixed in advance. The Adversary will choose some % < n 
such that, for many partial solutions PS of Pli, there is an extension of 
Pli to an instance A C Pli U Pi so that all valid/optimal/approximately 
optimal solutions to A are extensions of PS. We'll call such a partial solution 
indispensable, since if PS Ti, the Adversary can set Pi to A \ Pli so that 
the algorithm will never result in correct solution. 



3 Lower Bound of Knapsack 

First let us recall the definition of Knapsack problem. 
Knapsack Problem is defined by: 
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• Input: n pairs of non-negative integers, (x n ,p n ) and a positive 
integer N, Xi represents the weight of the ith item and Pi represents the 
value of the ith item. iV is the volume of the knapsack. 

• Output: S C {!,..., n}, such that J^i^sPi * s maximized with respect to 



Simple Knapsack Problem is defined by: 

• Input: n non-negative integers {xi, ...,x n } and a positive integer N, Xi is 
the weight and value of the ith item and N is the volume of the knapsack. 

• Output: S C {!,..., n}, such that J2i^s x i * s maximized with respect to 



Both problems are NP complete, we will consider only the simple Knapsack 
problem, and denote a simple Knapsack problem with n items and volume of 
N with (n, N). 

M.Alekhnovich et. al [1] have proved the following theorem: 

Theorem 1 For simple Knapsack problem (n,N), the time complexity of any 
adaptive BT algorithms is at least 



We will improve this lower bound using the same technique as the previous 
work, by optimizing related parameters, formally: 

Theorem 2 For simple Knapsack problem (n,N), the time complexity of any 
adaptive BT algorithms is at least 



where e is any small positive number. 

Proof: Consider positive numbers (3 > 7 satisfying (3 + 7 < 1, then 1 — /3 > 7, 
so we can choose another positive number a that satisfies a(l — (3) > 1 and 
0:7 < 1. These parameters will be fixed later to optimize the lower bound. 

Let N be some large integer, our initial set of items are integers in / = (0, a-—). 
Solver takes the first /3n items one by one, and following each one, Adversary 
applies the following rules to remove certain items from future consideration: 
remove all items that are the difference of the sums of two subsets already 
seen; also remove all items that complete any subset to exactly iV (ie all items 
with value iV — J2 x es x i where S is a subset of the items considered so far). 



Eies Xi < N. 
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These rules guarantee that at any point, no two subsets will generate the same 
sum, and that no subset will sum to N. Also notice that this eliminates at 
most 0(3 /3n ) numbers. To know why, the difference of the sums of any two 
subsets can be represented as a weighted sum of the numbers seen so far, with 
three possible weights: 1, and -1, so the number of distinct differences of any 
two subsets is at most 0(3 l3n )] the number of distinct values that complete any 
subset to exactly N is even less (0(2 /3n )), thus can be omitted here. So we will 
never exhaust the range from which Solver can pick the next item provided 
that N » 3" n . 

Call the set of numbers chosen so far P and consider any subset Q contained 
in P of size 771. Our goal is to show that Q is indispensable; that is, we want to 
construct a set R = Rq of size (1 — (3)n, consisting of numbers in the feasible 
input with the following properties. 

(1) Y,xeQuR x — N . 

(2) P U R does not contain other subsets that sum to N. 

The above properties indeed imply that Q is indispensable since obviously 
there is a unique solution with optimal value N and, in order to get it, Q is 
the subset that must be chosen among the elements of P. We thus get a lower 
bound on the width of the computation tree of any adaptive BT, which is the 
number of subsets of size 772 in P; namely (^")- Below is a diagram of such 
construction. 




Fig. 2. Construction of an indispensable partial solution 



For any Q C P, let U = 3 n , a = jj^y^N - E^qz), J — [a — U,a + U]. We 

show that J <Z I. Note that numbers in Q are less than a ■ — , so a — U > 
jY^iN - 7 n • a ■ f ) - 3 n = jtt^(N - 7 aiV) - 3 n > 0, the last step follows 

from 0:7 < 1 and iV >> 3 n ; and a + U < ■ N + 3 n < a ■ ^, the last step 

follows from a{\ - (3) > 1 and iV >> 3™. 

To construct R, first choose (1 — (3)n — 2 items in J. After each choice, we will 
remove some items from the remaining by the following rules. Let S be the 
set of items currently in P U R. 

(1) For all Si,S 2 C S, remove items of value | J^xeSx x ~ J2 X £S 2 x \- 

(2) For all Si C S, remove items of value iV — J2 x gSi x - 
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Since U = 3™, we can always avoid the points that need to be eliminated, and 
sum to a number w so that \w — a((l — f3)n — 2)| < U. This can be done by 
iteratively picking numbers bigger /smaller than a according to whether they 
average to below/above a currently. 

To complete we need to pick two more items 61,62 £ I that sum to v — 
(1 — j3)na — w, and so that they are not the difference of sums of any two 
subsets of the n — 2 items picked so far. Assume for simplicity that v/2 is an 
integer. Of the U + 1 pairs (v/2 — i,v/2 + i), where i = 1,2, U + 1, at least 
one pair (61, b 2 ) will suffice. 

(Note that, since \v— 2a \ < U, the smallest number is v/2 — U—l > a— ^ — 1 > 
^^-(iV - 7«iV) - ^ - 1 > 0, and the largest number is v/2 + U + 1 < 

a + ^ + 1 < jrzpy^ • N "+^ + 1 < a - ^, provided that N is sufficiently large. 
So we will never go beyond the range of I.) 

Now it has J2x^qur x = N, the only thing left is to prove there does not exist 
another subset which also sums to N. Suppose for contradiction S is such a 
subset, namely J2x^s x = N, following are three cases: 

(1) Neither b\ nor 6 2 belong to S, contradictary to the second rule when 
picking the first n — 2 items. 

(2) Both hi and b 2 belong to S, by comparing S and QUR, we get two subsets 
of the first n — 2 items that sum equally, contradictary to the first rule. 

(3) Exactly one of b\ and b 2 belong to S, by comparing S and Q U R, we find 
that bi or b 2 is equal to the difference of the sums of some two subsets of 
the first n — 2 items, contradiction. 

The rest of the proof is about maximizing the lower bound: (^j , subject to 
13 > 7 > and (3 + 7 < 1. 

According to Stirling formula, the target function can be simplified: 

f(3n\ {(3n)\ 



V2^i(^ 



(1) 
(2) 



v/2i7^(f )t» • y / 27r(/5-7)n(to^)(' 3 -^ 

By applying logorithm transform, our goal is equivalent to maximize: 

f({3, 7) = Plnp - 7 /n 7 - (0 - 7 )Zn(/3 - 7), 
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subject to (3 > 7 > and (3 + 7 < 1. 



Note that for convenience, we allow (3 + 7 = 1 here, the case will be handled 
later. Take partial differential on (3, which is: 

?I^ = ln(1-ln([l- 1 )>0 



So /(/3,7) is strictly increasing respect to /3, since /3 < 1 — 7, by setting (3 to 
1 — 7, we only need to maximize: 

9{j) = /(l - 7, 7) = (1 - l)ln{l - 7) - 7^7 - (1 - 2 7 )M1 " 27) 

Whose differential is: 

#'(7) = 2Zra(l - 27) - Zn 7 - Zn(l - 7) = Ir S^ ~ ^ 

7(1-7) 

The only root of the above function that lies in (0, |) is 5 ~^ , which maximizes 
g, thus /. So /3 can be arbitrarily close to 1 — 7 = , and the maximum 
value of y7Tg~p=^ i s 1-618 — e by substitution, where e is any small positive 
number. Consequently, the optimal lower bound f^j is: 

(1.618 - e)7v^ = 2(°- 694 - £ ) n /v^ 



4 Discussion and Future Work 



Backtracking is an important algorithmic scheme which is pervasive in solving 
hard problems such as Propositional Satisfiability Problem (SAT) and Con- 
straint Satisfaction Problem (CSP). Proving lower bounds for these problems 
under BT model will both deepen our understanding of the structure and prop- 
erties of these hard problems and guide our designing of algorithms to solve 
these problems. An exponential time lower bound for SAT under fully adap- 
tive BT model has already appeared in pp. Recently, Xu and Li have proven 
an exponential lower bound for a class of random CSPs (Model RB) under 
tree-like resolution [5]. Since tree-like resolution has a close tie with DPLL, a 
famous backtracking search strategy for SAT with corresponding exponential 
time lower bound in [I], it is natural to investigate the time complexity of 
some class of random CSPs (RB model) under BT model, and we expect that 
this future work may produce the first exponential time lower bound for some 
class of random CSPs under BT model. 
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